IEVQ: An Iterative Example-Based Visual Query for Pathology Database

Microscopic image analysis of nuclei in pathology images generates tremendous amount of spatially derived data to support biomedical research and potential diagnosis. Such spatial data can be managed by traditional SQL based spatial databases and queried by SQL for spatial relationships. However, traditional spatial databases are designed for structured data with limited expressibility, which is difficult to support queries for complex visual patterns. Moreover, SQL based queries are not intuitive for biomedical researchers or pathologists. In this paper, we investigate the expressive power of visual query for spatial databases and propose an effective yet general Iterative Examplebased Visual Query (IEVQuery) framework to query shapes and distributions. More specifically, we extract features from nuclei in pathology databases, such as shape polygon nuclei density distribution, and nuclei growth directions to build search indexes. The user employs visual interactions such as sketching to input queries for interesting patterns. Meanwhile, the user is allowed to iteratively create queries, which are based on previous search results, to finely tune the features more accurately to find preferred results. We build a system to enable users to specify sketch based queries interactively for 1) nuclei shapes, 2) nuclei densities, and 3) nuclei growth directions. To validate our methods, we take a pathology database [11] consisting of hundreds of millions of nuclei, and enable the user to search in the database to find most matching results through our system.